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Abstract 

This paper examines whether a single first-year inquiry-based seminar can have a lasting impact on students’ 
academic skills. Fifty-four Inquiry students and 71 comparable students participated in three performance 
tests: a research skills exercise; an evaluation of oral presentation ability; and a test of critical reasoning and 
teamwork skills. In addition, participants completed a questionnaire focusing on learning approaches and 
experiences. The study demonstrates the feasibility of measuring long-term effects of relatively small 
educational interventions. Findings indicate that although universities may not be developing the skills they 
assume, a single first-year inquiry seminar can have far reaching effects on academic skill development and 
these skills are typically lasting. 
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Abstract 

This paper examines whether a single first-year inquiry-based seminar can have a lasting 
impact on students' academic skills. Fifty-four Inquiry students and 71 comparable 
students participated in three performance tests: a research skills exercise; an evaluation 
of oral presentation ability; and a test of critical reasoning and teamwork skills. In addition, 
participants completed a questionnaire focusing on learning approaches and experiences. 
The study demonstrates the feasibility of measuring long-term effects of relatively small 
educational interventions. Findings indicate that although universities may not be 
developing the skills they assume, a single first-year inquiry seminar can have far reaching 
effects on academic skill development and these skills are typically lasting 

Keywords: Inquiry; inquiry-based learning; quasi-experimental study; academic skills; 
first-year seminar 


Introduction 

The last twenty years has seen increased criticisms of higher educational practice and calls 
to reform undergraduate programs (e.g. Barr and Tagg, 1995, Boyer Commission, 1998, 

Bok, 2006). Universities have introduced approaches such as problem-based, experiential, 
service, and inquiry-based learning to engage students in the learning process and build 
academic skills. Barr and Tagg (1995) claimed these new innovations represent a paradigm 
shift from an instructional focus to an approach centered on active engagement and self- 
directed learning. There has also been a growth in first-year seminar courses focused 
variously on easing the transition to higher education, introducing interdisciplinary themes, 
developing study and life management skills, and teaching intellectual skills necessary for 
academic success (Porter and Swing, 2006). In the United States, 90% of colleges and 
universities offer such first-year seminars to some of their students (Barefoot, 2002). 

In order that resources are not wasted on developing courses which have unachievable goals 
in the time allotted (Bok 2006: 279), curricular design should be based not simply in worthy 
aims but also in evidence of interventional effectiveness. The shift in pedagogy from 
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instructor led to self directed learning has raised question about the efficacy of moving in 
this direction. Van Gelder describes how difficult it is to help students master critical 
thinking skills. He claims it is about as difficult as learning a new language requiring 
considerable time and effort (van Gelder, 2005: 42). 

While difficult to generalize about the impact of individual first-year seminars, they have 
been linked to improved retention and improved academic performance measured by GPA 
and related indicators (e.g.; Fidler, 1991, Williford et. al., 2000-01, Porter and Swing, 2006, 
Justice et al, 2007). These measures have not explored whether the move to a new 
pedagogy has contributed to building intellectual and academic skills, or whether effects 
are significant and lasting -- a noteworthy question given the consensus that skills are best 
developed when integrated into substantive curricula as opposed to in separate modules 
(Jenkins & Pepper 1988; Shepherd 2000). Likewise, the effectiveness of inquiry pedagogy 
in enhancing learning and in producing lifelong learners is largely unexamined, as is the 
pedagogical significance of linking research and teaching for which inquiry learning may 
be an important modality (Brew, 1999, Healey, 2005). 

The question asked in this paper is: To what extent can a single first-year inquiry 
course provide students with academic skills and intellectual habits that will 
positively impact on their ability to learn? To answer this question, we describe a 
quasi-experimental research project measuring the long-term effects (up to 5 years) on 
students' involvement in a first-year experimental inquiry course offered during the period 
1998-2002 at McMaster University. During this time 768 first-year social science or 
kinesiology students enrolled in Inquiry 1SS3. A preliminary 2002 study of students' 
academic records comparing Inquiry and non-inquiry students suggested the seminar had a 
positive effect on academic performance (Justice, Rice, Warry, & Laurie, 2007) as judged 
by grades and likelihood of finishing a degree. In order to examine these preliminary 
findings more deeply, a comprehensive research program was developed in 2004. The 
McMaster Innovative Learning Study (MILS) was established to examine the ways Inquiry 
students differed from non-inquiry students and how these findings might explain 
differences in academic performance. This paper describes key findings of this study and 
explores the longer-term consequences associated with taking the inquiry seminar. 


Inquiry - Pedagogical Objectives 

The intent of Inquiry 1SS3 was to encourage students to develop intellectual and academic 
skills which we hypothesize leads to active self-directed learners, with life-long learning 
skills who approach knowledge and understanding deeply, critically and reflectively. 

Figure 1 sets out the course objectives. 


Figure 1: Inquiry 1SS3 Course Objectives 
Students were expected to develop their capacity in the following areas: 

• being actively engaged and self-directed in academic research by: 
o engaging a topic 

o developing a good question 

o anticipating the information needed to answer this question 
o accessing and evaluating information 

o synthesizing information into knowledge 

o communicating the inquiry process and new knowledge 
o evaluating success 

• enjoying the learning process 
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• developing a deep approach to learning 

• thinking critically 

• reflecting on and evaluating their process as a learner 

• learning in collaboration with others 

• communicating effectively orally and in writing 

• exploring the topical issues of social identity and representation 


The Inquiry course design, learning exercises, assessment methods, and guidance given to 
instructors, are described in two previous publications (Justice, Warry, Cuneo, Inglis, Miller, 
Rice & Sammon. 2002; Justice, Rice, Warry, Inglis, Miller & Sammon 2007). Here, we 
summarise the pedagogical logic and those features of the course which are critical to 
interpreting the results of this study. 

The pedagogical theory underlying Inquiry 1SS3 is that students will be more successful in a 
learning 'ecology' which has four critical features. First, students must be encouraged to 
explore compelling questions of their own design, with instructors animating this process, 
acting as resource people and skill developers. Second, necessary intellectual and academic 
skills need to be introduced at a time when they are required by students to further their 
inquiries. Third, the environment must be supportive of open-minded discussions, where 
students are encouraged to question their assumptions and critically assess information. 
Fourth, students require regular feedback and need to be graded in ways relevant to the 
skills and abilities underlying the inquiry process. 

Inquiry 1SS3 was a limited-enrolment course taught in sections of 20 to 30 students. 
Sections used a common schedule, reading material, and process of assessment. The 
classes met for twelve three-hour concurrent evening sessions. All sections investigated 
aspects of a social science thematic question related to representation of Others. Within this 
theme, students developed their own individual questions. Weekly classes involved students 
learning to investigate their questions through the inquiry process represented in Figure 2. 
Class activities included instructor-guided skills exercises coinciding with the staged 
development of students' investigations. Skill development tasks were conducted in 
participatory learning groups of four or five students. 


Figure 2: The Inquiry Process (from Justice et al. 2007) 


Taking 
resp onsibility 
for learn ing 


Enga ging a topic 
build ing basic 
kno wledge 

Evaluating Dev eloping a ques tion 

succ ess 


A 

Com municating new 
und erstandings 


Self reflection 
& 

Self eva luation 


Anticipating po ssible 
answ ers and deter mining 
relev ant information 

y 


Weighing evidence and 
synthesizing unde rstandings 


T 


Ass essing 
infor mation 


Iden tifying resources and 
gathering infor mation 


https://doi.org/l0.2042 9/ijsotl.2009.030109 


3 




Academic Skill Development 


Method of McMaster Innovative Learning Study 

Participants 

We offered students $100.00 to participate in the study. Our goal was to recruit a sample of 
80 participants who had taken Inquiry 1SS3 and 80 who had not (from the potential pool of 
6244 candidates who had the opportunity to take Inquiry 1SS3 between 1998 and 2002). 
Students were told only that the study concerned innovative learning. In the end, 54 
students who had had taken Inquiry 1SS3 (referred to as 'Inquiry students' or 'Inquiry 
group') and 71 students who had not taken Inquiry 1SS3 ('Non-Inquiry students' or 
'Comparison group') completed the study. 

We came to this study well aware of the complexities of conducting research on the living 
world. A true experiment was neither possible for ethical or practical reasons nor desirable 
as the presence of disinterested students would change the educational environment. As 
students were not randomly assigned to the intervention, any measured differences 
between Inquiry and Non-Inquiry students are potentially related to either the effects of the 
course or to pre-existing differences between groups related to the self-selection process. 

We have approached this complexity by: 1) considering the potential for self-selection bias 
very thoroughly in the planning, execution, analysis and interpretation of the research; 2) 
clearly acknowledging that student self-selection creates concerns that there are unknown 
differences between groups including motivation ; 3) testing for these differences comparing 
the groups in all dimensions for which we have data; and 4) drawing inferences about the 
importance of potential pre- intervention group differences from the study results including 
questionnaire data and patterns of skill differences. 

We tested several variables to see if there were differences between the study groups. As 
presented in Table 1, there are no meaningful differences between the study groups in 
gender, age or high school performance, nor in reported parents' income, student loans, or 
the highest degree student's had earned at the time of study. We cannot directly answer the 
important question of whether students who chose ISS3 were more academically motivated 
or not. However, any motivational difference is not reflected in either high school or first- 
year university performance. Given the similarity between the two groups we felt the 
differences in performance between 1SS3 students and the comparison group were worth 
exploring in detail. 


Table 1. Comparison of variables between Inquiry and Non-Inquiry Students 


Level 

Means 


% Female 

Age 

HS English 

HS GPA 

Level 1 

Non-Inquiry 

82 

19.2 

77.0 

79.5 

Avg. 

6.9 

Inquiry 

74 

19.1 

79.3 

79.8 

7.0 

Total 

78 

19.2 

78.1 

79.6 

6.9 


Instruments 

We transformed course objectives (Figure 1) into output indicators to measure student's 
skills through three sets of exercises, each completed in a two hour period during a single 
day workshop. The first exercise tested students' academic research skills, including critical 
thinking and reading skills, question development, research planning, and information 
gathering. The second exercise evaluated student's videotaped presentations to a small 
audience. The third examined critical reasoning and teamwork ability. In addition, 
participants completed an online questionnaire exploring approaches to learning, academic 
and social activities, attitudes, perceptions, satisfaction, and self-perceived change during 
university. 
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Analysis Procedures 

To ensure objective assessment of students' performance, multiple blinded Raters who were 
educators from outside the university or extensively trained graduate students assessed the 
results of the exercises. Differences between study groups were compared using analysis of 
variance procedures. The main thrust of our statistical analysis is the calculation of effect 
size, a measure of the magnitude or importance of a treatment effect which is independent 
of the size of the sample using Cohens d (Cohen,1988). We also calculated probability value 
in the standard way, but this is not meant as a stand-alone test of differences. Because we 
were testing for the effects of a single course in the context of as many as 40 courses and 
after as many as five years, we did not expect to see large effects. In addition, because our 
analyses needed to control for university level, the added variance related to multiple 
instructors, and our small sample size, we expected to see low effect sizes and relatively 
high p. values. Though there are many results which are statistically significant according to 
the classical criteria of p<0.05, we decided to accept differences as meaningful if they have 
a minimum effect size of 0.2 (which Cohen considers to be a small effect) in combination 
with a reasonable probability value which, we judge to be 0.2. 

Because we were testing skills generally developed at university, we also controlled for the 
number of courses a student had taken as a measure of time spent at university or 
university exposure. Students were grouped dichotomously into Lower-Level students (who 
had completed fewer than 84 units) and Upper-Level students (more than 84 units). Lower- 
Level students, on average, had about half the university experience compared to Upper- 
Level students: a difference of 53.2 units, or just less than two full years. 


Table 2. Comparison of variables between levels 


Level 

Means 


% Female 

Age* 

HS English 

HS GPA 

Level 1 Avg 

Lower Level 

78 

19.1 

77.4 

79.4 

6.7 

Upper Level 

79 

19.2 

78.6 

79.9 

7.2 

Total 

78 

19.2 

78 

79.6 

6.9 


* Age when course was taken 


As there are no cohort differences between dichotomous level groups (Table 2), we believe 
it is reasonable to conclude that when we compare skill levels between Lower and Upper- 
Level students (controlling for study group) we are seeing the effects of time spent at 
university. In the tables that follow, we compare performance differences between study 
groups with those between Lower-and Upper- Level students. This provides a sense of the 
relative strength of the effect of being in one or the other study group to the effect of being 
in university for two full years. 

In the following tables, scores are reported as estimated marginal means scaled to 
percentiles. Bolded results are those we consider meaningful with effect size higher than 
0.20 and p value of less than 0.20. Figures with Source labeled Group or Level are 
comparisons by study group and university level respectively. Figures with Source labeled 
Lower or Upper represent interactions of group and level. 


Questionnaire Results 

The questionnaire consisted of 140 questions related to approaches, attitudes to learning 
and experiences of university. Questions were designed to either discover any pre-existing 
differences between groups that might alternatively explain study effects or to measure 
change associated with having taken the Inquiry seminar. 
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The Revised Study Process Questionnaire Biggs, Kember & Leung, (2001) examines deep 
and surface approaches to learning. We found no main effects either between the Inquiry 
group and the Comparison group or between Lower and Upper-Level students. This 
suggests that neither being in the Inquiry seminar nor time spent at university has much 
of an impact on students' approach to learning. However, among Non Inquiry group more 
time spent at university is associated with a decrease in deep approach to learning 
suggesting the Inquiry seminar provided a small protective effect against a more general 
decrease in deep approach to learning. 


Table 3. Life Long Learning 


Variable 

Source 

Estimated Marginal 
Means 

Non- Inquiry 

Inquiry 

Effect 

size 

AN OVA 

F p. 

Making Sense of Course 

Group 

68.1 

67.8 

.036 

.040 

.842 

Material 


Lower 

67.4 

66.6 

.072 

.128 

.721 


Upper 

68.8 

69.0 

.020 

.128 

.721 

Being Self-Directed 

Group 

79.5 

80.4 

.088 

.192 

.662 


Lower 

80.4 

83.8 

.246 

1.688 

.196 


Upper 

78.7 

77.0 

.122 

1.688 

.196 

Being Guided by Others 

Group 

51.0 

47.9 

.308 

1.424 

.235 


Lower 

53.5 

45.5 

.574 

3.826 

.053 


Upper 

48.4 

Lower 

Level 

50.3 

Upper 

Level 

.139 

3.826 

.053 

Making Sense of Course 

Level 

67.0 

68.9 

.210 

1.669 

.199 

Material 

Being Self-Directed 

Level 

82.1 

77.8 

.388 

4.746 

.031 

Being Guided by Others 

Level 

49.5 

49.4 

.015 

.004 

.949 


The Life Long Learning Scale (Knapper and Cropley 2000) showed a similar pattern. There 
were no main effects related to being in the Inquiry group nor to time spent in university. 
However, a factor analysis revealed three components: Making Sense of Course Material, 
Being Self-Directed and Being Guided by Others (Table 3), and we found meaningful effects 
in the component Being Self-Directed. Lower-Level students score higher than Upper-Level 
students suggesting that self-perception of being self-directed diminishes with time spent in 
university. Among Lower-Level students only, Inquiry students score meaningfully higher in 
Being Self-Directed than the Non-Inquiry students and lower in Being Guided by Others. 
This suggests that Inquiry 1SS3 had some positive effect on the self-perception of being a 
self-directed learner and a positive effect against the self-perception of being guided by 
others. However, this effect does not persist, perhaps because the university environment 
has an overall negative effect on the self-perception of development of life-long learning 
traits. 

Other questionnaire items pertained to experiences in and engagement in both academic 
and non-academic activities. Results are shown in Table 4. 
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Table 4. Engagement and Experience Questionnaire Results _ 

Variable Estimated Marginal ANOVA 

Means 



Source 

Non- 

Inouirv 

Inquiry 

Effect 

F 

D. 




size 



Engagement 

Group 






Scholastic Engagement 


53.1 

52.6 

.048 

.070 

.792 

Scholastic Disengagement 


46.6 

42.4 

.215 

1.396 

.240 

Broad Scholastic Interest 


25.7 

29.9 

.246 

1.815 

.180 

Academic Community 
Engagement 


28.2 

31.9 

.196 

1.153 

.285 

Engagement with Professors 


27.7 

28.7 

.032 

.030 

.862 

Involvement in University life 


50.7 

51.6 

.076 

.176 

.676 

Work and Family Activities 


35.0 

27.6 

.354 

3.763 

.055 

Rating of Experience 







Satisfaction with University 


62.6 

61.9 

.049 

.071 

.790 

Skill development at University 


77.9 

83.3 

.713 

15.309 

.000 

Successful socialization at 
University 

Would you still choose 


75.9 

74.2 

.123 

.452 

.503 

McMaster? 


82.0 

87.0 

.269 

2.085 

.151 



Lower 

Upper 






Level 

Level 




Engagement 







Scholastic Engagement 

Level 

49.8 

55.9 

.546 

9.374 

.003 

Scholastic Disengagement 


46.6 

42.4 

.215 

1.455 

.230 

Broad Scholastic Interest 


26.1 

29.5 

.197 

1.220 

.272 

Academic Community 
Enqaqement 


29.7 

30.3 

.031 

.031 

.861 

Engagement with Professors 


18.4 

38.0 

.601 

11.362 

.001 

Involvement in University life 


49.1 

53.1 

.328 

3.378 

.069 

Work and Family Activities 


29.8 

32.8 

.143 

.640 

.425 

Rating of Experience 







Satisfaction with University 

Skill development at Universit i 
Successtul socialization at 
University 

wouiq you still choose 

McMaster? 


60.7 

78.9 

72.3 

63.9 

82.3 

77.8 

.209 

.435 

.408 

1.378 

5.943 

5.242 

.243 

.016 

.024 






82.9 

86.1 

.172 

.890 

.348 


As would be expected, time spent at university has a number of independent effects. 
Upper-Level students report higher levels of scholastic engagement, involvement in 
university activities and increased connection to professors. These students, more than 
Lower-level students, also perceive themselves as having succeeded at university and 
as having developed more skills. 
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By and large, there are few meaningful differences between the Inquiry group and the 
Comparison group in these areas. Inquiry group students report fewer part-time jobs and 
family responsibilities that interfere with studying. Perhaps correspondingly, Inquiry 
students reported higher levels of indicators of broad scholastic interest. Given their 
university experience, Inquiry students were also more likely to say that they would still 
choose McMaster again. In another question about general preference to work 
independently or as a part of a team, 39% of the Inquiry group said they prefer teamwork 
against 26% of the Comparison group. 


Table 5. Perceived development of skills while at university 


Variable 

Source 

Estimated 
Marginal Means 

Non- T 

T Inquiry 

Inquiry ' 

Effect 

size 

AN OVA 

F p. 

Knowledge and Academic Skills 

Group 

85.8 

90.2 

.564 

9.578 

.002 

Social Awareness 


76.7 

82.6 

.470 

6.662 

.011 

Interpersonal skills 


73.4 

81.8 

.634 

12.098 

.001 

Understanding other cultures 


71.0 

75.1 

.388 

4.526 

.035 

Computer skills 


75.7 

80.6 

.340 

3.472 

.065 



Lower 

Upper 






Level 

Level 




Knowledge and Academic Skills 

Level 

86.4 

89.6 

.406 

5.192 

.024 

Social Awareness 


78.2 

81.2 

.237 

1.760 

.187 

Interpersonal skills 


74.1 

81.1 

.516 

8.361 

.005 

Understanding other cultures 


72.7 

73.4 

.067 

.142 

.707 

Computer skills 


75.8 

80.5 

.315 

3.130 

.079 


There is, however, one area of striking difference between Inquiry and Non-Inquiry students 
in the questionnaire data: student perceptions of their development in 14 skills. Factor 
analysis resulted in five components, presented in Table 5. 

As would be expected, there is a significant effect related to time spent at university. Upper- 
Level students report greater skill development than Lower-Level students in all components 
except one related to Understanding Other Cultures. Remarkably, Inquiry students report 
greater skill development than do the Non-Inquiry students in all components and with 
effect sizes more substantial than that between Upper and Lower-Level students. As the 
difference between groups does not diminish with time spent at university this evidence 
suggests that the effects of the course on the way students recognize their academic skills 
were persistent. This area of difference between study groups is logically related to the 
Inquiry seminar which explicitly dealt with skill development and skill recognition. 

Overall, the questionnaire responses suggest Inquiry students are not much different from 
the comparison group when it comes to their experiences of university, their approaches to 
learning, and in the way they value education. However, compared to other students, 

Inquiry students believe they have changed in fundamental ways: they believe they have 
developed greater social awareness, better interpersonal skills, and more understanding 
of diversity. They also believe they have developed more in both knowledge and academic 
skills. But have they? 
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Exercise 1 - Academic Research Skills 

This set of four linked tasks conducted over a two hour period tested academic research 
skills. Students were asked to 1) read and summarize an academic paper, 2) design a 
question and a research plan, 3) find good information relevant to the question, and 4) 
carry out a self-evaluation of performance and ability. This material was graded by three 
trained Raters who were blind to the study design. Raters used a validated scoring sheet 
adapted from Trim (2006) to score each exercise. Raters assessed the critical thinking 
exhibited in the first two exercises and the quality of the four exercises as a whole. Inter- 
Rater reliability was acceptable (Cronbach's Alpha = .878). 

Component 1 - Reading and Summarizing Material 

Students were given a reasonably short, complex and controversial research article (Duncan 
2003) and asked to write a 200 word summary in a 40-minute period. Raters assessed 
these summaries for their clarity, accuracy and apparent understanding of the research 
paper {Clarity and Accuracy of Summary) and holistically rated the overall summary quality. 
The results are presented in Table 6. 

For both measures, Raters judged Inquiry students' work more highly than that of the 
Comparison group. For the factor Clarity and Accuracy of Summary the difference between 
study groups is bigger than that associated with time spent in university. This skill 
advantage does not diminish with time. 


Table 6. Quality of article summaries 


Variable 

Source 

Estimated Marginal 
Means 

Non- Inquiry 

Inquiry 

Effect 

size 

AN OVA 

F 

P- 

Overall quality of Summary 

Group 

62.4 

66.1 

.259 

2.041 

.154 

Clarity and Accuracy of 
Summary 


66.8 

72.4 

.306 

8.346 

.004 

Overall quality of Summary 

Level 

Lower 

Level 

61.4 

Upper 

Level 

67.1 

.396 

4.890 

.028 

Clarity and Accuracy of 
Summary 


67.8 

71.5 

.199 

3.694 

.055 


Raters also assessed student summaries on the basis of their critical intentionality or what 
aspects of the research paper students chose to focus on. The analysis revealed three 
factors: Focusing on Factual Details, Focusing on Author's Argument, and Focusing on 
Research Methods. The results are presented in Table 7. Time spent at university is 
associated with higher scores related to focusing on methodology and the factual details 
of the article. Lower-Level Inquiry students were more likely to focus their summaries on 
the factual details and Upper-Level Inquiry students were less likely to focus on 
methodology. This assessment of critical intentionality shows Inquiry students approach 
the writing of summaries differently than the Comparison group and suggests that better 
capturing of factual details is associated with higher summary ratings. 
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Table 7. Critical intentionality of summaries 


Variable 

Source 

Estimated Marginal 
Means 

Non- Inquiry 

Inquiry 

Effect 

size 

AN OVA 

F 

P- 

Focusing on Factual Details 

Group 

63.5 

68.4 

.243 

5.258 

0.022 


Lower 

58.2 

69.1 

.536 

7.632 

0.006 


Upper 

68.7 

67.7 

.050 

7.632 

0.006 

Focusing on Author's 

Argument 

Group 

18.7 

22.0 

.145 

1.869 

0.172 


Lower 

17.5 

22.1 

.199 

0.260 

0.611 


Upper 

19.9 

22.0 

.092 

0.260 

0.611 

Focusing on Research Methods 

Group 

42.1 

39.4 

.131 

1.536 

0.216 


Lower 

37.6 

38.0 

.015 

1.912 

0.168 


Upper 

46.5 

40.9 

.282 

1.912 

0.168 

Focusing on Factual Details 

Level 

Lower 

Level 

63.7 

Upper 

Level 

68.2 

.220 

4.523 

0.034 

Focusing on Author's 

Argument 

Level 

19.8 

20.9 

.047 

0.203 

0.652 

Focusing on Research Methods 

Level 

37.8 

43.7 

.284 

7.535 

0.006 


Component 2 - Research Project Design 

Students were asked to design research they would do related to the article they had just 
summarized. They were asked to write down questions or ideas related to the article, to 
develop one question suitable for research, to explain why this question is appropriate, and 
finally to outline a realistic research plan for their question, identifying both specific 
information and procedures for finding evidence. Raters scored the material in two areas 
1) apparent ability to design a good research question and; 2) ability to outline a good 
research plan. 

Raters judged the overall quality of the research question (Table 8), both using their own 
subjective considerations and by attributing 'desirable qualities' to the question from a 
supplied list. They also rated justifications of why the question was appropriate. The list of 
desirable qualities included analytical approach; focus on a dilemma or problem; openness 
to possible answers; theoretical foundation; relevance to the original paper, creativity; 
research ability; and social importance. In the assessment of question quality, Lower-Level 
Inquiry students received meaningfully higher scores than did the comparisons. However, 
for Upper-Level students the Non-Inquiry group received higher scores. In counting 
desirable qualities, Inquiry students' questions had fewer desirable qualities than those of 
the comparisons but at the Upper-Level only. Finally, Inquiry students offer significantly 
fewer good reasons why their question is appropriate than the Non-Inquiry group. These 
mixed results suggest an initial, small, but non-persistent advantage to inquiry students 
in the ability to design a good question. However, overall Inquiry is associated with a net 
disadvantage in ability to design or describe a research question. 
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Table 8. Question Quality and Justification 


Variable 

source 

Estimated Marginal 
Means 

Non- inquiry 

Inquiry 

httect 

size 

ANOVA 

F p. 

Number of desirable qualities 

Group 

43.3 

40.0 

.316 

3.026 

.083 

of research question 

Lower 

40.3 

40.8 

.048 

4.035 

.045 


Upper 

46.3 

39.2 

.700 

4.035 

.045 

Overall quality of the research 

Group 

62.7 

60.5 

.151 

0.700 

.403 

question. 

Lower 

57.3 

61.8 

.313 

6.490 

.011 


Upper 

68.2 

59.2 

.612 

6.490 

.011 

Number of good reasons given 

Group 

8.4 

5.5 

.454 

6.291 

.013 

why question is appropriate 

Lower 

7.7 

4.2 

.546 

0.209 

.648 


Upper 

9.1 

6.7 

.373 

0.209 

.648 

Overall quality of justification 

Group 

55.4 

55.5 

.004 

.000 

.984 

of question appropriateness 

Lower 

53.9 

50.2 

.239 

1.809 

.180 


Upper 

56.9 

Lower 

Level 

60.7 

Upper 

Level 

.248 

1.809 

.180 

Number of desirable qualities 
of research question 

Level 

40.5 

43.8 

.212 

1.392 

.239 

Overall quality of the research 
question. 

Level 

59.5 

63.7 

.282 

2.458 

.118 

Number of good reasons given 
why question is appropriate 

Level 

6.0 

7.9 

.292 

2.653 

.104 

Overall quality of justification 
of question appropriateness 

Level 

52.1 

58.8 

.428 

5.746 

.017 


Research Planning is represented by a summation of Raters' scores (Table 9) in four areas: 
1) plan specificity, 2) plan practicality, 3) likelihood of resulting in good data, and 
4) familiarity with diverse methods. There is a slight improvement in this skill with time 
spent at university. There is no overall group difference between Inquiry students and the 
Comparison group in research planning. However, there is a significant interaction effect. 
Lower-Level Inquiry students score higher at planning research than the Lower-Level Non- 
Inquiry group, but this pattern is reversed among Upper-Level students where the 
comparison group scored higher. 
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Table 9. Student's Research Plan 


Variable 

Source 

Estimated Marginal 
Means 

Non- Inquiry 

Inquiry 

Effect 

size 

AN OVA 

F p. 

Quality of the Student's 

Group 

55.3 

54.4 

.074 

0.164 

.685 

research plan (summary of 
four areas) 

Lower 

50.2 

56.6 

.542 

11.274 

.001 

Upper 

60.4 

Lower 

Level 

52.2 

Upper 

Level 

.680 

11.274 

.001 

Quality of the Student's 
research plan(summary of 
four areas) 

Level 

53.4 

56.3 

.242 

1.824 

.178 


Component 3 -Critical Thinking 

To judge critical thinking, Raters considered both the article summaries and the research 
plan described above. Raters looked for evidence of four critical activities: 1) conveying 
meaning in the words and concepts (interpretation), 2) conveying comprehension of the 
overall meaning (analysis), 3) questioning whether the text makes sense or whether the 
findings are valid or reliable (evaluation); and 4) drawing conclusions from the student's 
understanding of the text or making connections to previous knowledge (inference). A 
second aspect of the assessment of critical thinking was Raters' subjective assessment of 
the student's apparent ability to think critically. There are no meaningful differences 
between Inquiry group and the Comparison group and only a small positive effect between 
Upper and Lower-Levels in one measure of critical thinking suggesting university experience 
has a small beneficial effect on critical thinking. 


Table 10. Critical thinking in article summarizing and research planning 


Variable 

bource 

Estimated Marginal 
Means 

Non- inquiry 

Inquiry 

trrect 

size 

ANOVA 

F p. 

Critical thinking (summary of 

Group 

62.2 

58.5 

.234 

1.643 

.201 

four questions) 

Lower 

61.8 

54.9 

.437 

1.204 

.273 


Upper 

62.6 

62.0 

.033 

1.204 

.273 

Assessment of critical 

Group 

69.3 

67.9 

.098 

0.358 

.550 

thinking ability 

Lower 

68.6 

65.5 

.228 

0.421 

.517 


Upper 

70.1 

70.2 

.009 

0.421 

.517 



Lower 

Upper 






Level 

Level 




Critical thinking (summary of 

Level 

58.3 

62.3 

.246 

1.884 

.171 

four questions) 







Assessment of critical 

Level 

70.1 

67.0 

.217 

1.469 

.226 

thinking ability 








Component 4 - Accessing Information 

Following the research planning exercise, students were given a set question and asked to 
find quality information to help them answer this question. They were asked to list as many 
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relevant and useful articles, books and websites as they could find, and to identify the 3 
journal articles, 3 academic books and 3 trustworthy (non journal) web sites that they found 
the most useful. Raters assessed both the students'abilities at locating useful information 
and the process as a whole. (Table 11). 

Table 11. Accessing Information Results _ 

Variable Estimated Marginal ANOVA 

Means 



Source 

Non- 

Inquiry 

Inquiry 

Effect 

size 

F 

P- 

Accessing Journals 

Group 

51.3 

58.4 

.503 

7.638 

.006 


Lower 

40.4 

54.8 

1.057 

8.487 

.004 


Upper 

62.3 

61.9 

.027 

8.487 

.004 

Accessing Books 

Group 

47.5 

47.2 

.017 

0.008 

.928 


Lower 

41.2 

47.6 

.420 

5.667 

.018 


Upper 

53.8 

46.9 

.447 

5.667 

.018 

Accessing Web Material 

Group 

47.5 

45.2 

.152 

0.701 

.402 


Lower 

45.9 

41.2 

.308 

0.709 

.400 


Upper 

49.2 

49.2 

.001 

0.709 

.400 

Holistic Rating of Information 

Group 

51.9 

52.1 

.014 

.006 

.937 

Accessing 

Lower 

44.7 

49.7 

.433 

5.140 

.024 


Upper 

59.1 

54.4 

.396 

5.140 

.024 

Accessing Journals 

Level 

Lower 

Level 

47.6 

Upper 

Level 

62.1 

1.020 

32.411 

.000 

Accessing Books 

Level 

44.4 

50.3 

.382 

4.549 

.034 

Accessing Web Material 

Level 

43.5 

49.2 

.363 

4.112 

.043 

Holistic Rating of Information 

Level 

47.2 

56.8 

.809 

20.172 

.000 


Accessing 


Upper-Level students score significantly higher than Lower-Level students in all aspects of 
our measures of accessing information, indicating that this is a skill set that students 
improve upon in attending university. Lower-Level Inquiry students score higher than 
Lower-Level Non-Inquiry students on accessing journal articles. It seems that Inquiry 
students are given an advantage in this area and, though they continue to improve with 
time, the Comparison group students catch up by Upper-Level. While taking Inquiry 1SS3 
has no effect overall on accessing books, there is an interaction with level. Lower-Level 
Inquiry students have an advantage but they do not seem to progress and Upper-Level 
Inquiry students are at a disadvantage. There are no group differences in accessing useful 
websites. 

These results make sense given the Inquiry seminar training, which focused primarily on 
finding journal articles, only somewhat on books, and with the assessing, but not the 
finding, of web-based material. We take the overall pattern of these results as evidence that 
effects more generally can be linked to the course and not to pre-existing group differences. 
The Holistic Rating of Information Accessing shows a similar pattern to question 
development and research planning; Inquiry students have an advantage in the first couple 
of years after taking the seminar, but in later years they score less well than Non-Inquiry 
students. 
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Component 5 - Self-assessment 

In the final research skills task, students were asked evaluate their strengths and need for 
improvement in the previous areas. Raters examined student self-assessments and scored 
them according to their willingness to self-assess, their open-mindedness to strengths and 
weaknesses, their analytical self-confidence, and their inquisitiveness regarding multiple 
points of view. These responses were summed to create a component Student Self- 
Assessment. In addition, the Raters provided a holistic impression of self assessments, 
referred to as Overall Rating. 


Table 12. Self-assessment 


Variable 

Source 

Estimated Marginal 
Means 

Non- Inquiry 

Inquiry 

Effect 

size 

AN OVA 

F p. 

Student Self-Assessment 

Overall Rating 

Group 

Group 

56.6 

59.0 

59.4 

63.6 

.291 

.382 

2.544 

4.41 

.112 

.036 

Student Self-Assessment 

Overall Rating 

Level 

Level 

Lower 

Level 

56.6 

59.5 

Upper 

Level 

59.4 

63.2 

.286 

.304 

2.549 

2.907 

.111 

.089 


Inquiry students were judged to be better at self-evaluation relative to the Comparison 
group in both measures and with a magnitude similar to the difference between Upper-Level 
and Lower-Level students, roughly equivalent to two full years of university experience. 

Component 6 - Holistic Assessment of Academic Research Skills 

Finally, Raters considered their overall impression of the academic research skills exercises 
not taking into account the component scores for the four parts of the exercise. Despite the 
mixed results for some of the research exercise tasks when rated as individual components, 
when Raters judged research skills holistically Inquiry was associated with higher overall 
scores with a medium effect size. The magnitude of this advantage is similar to that 
between Upper-Level and Lower-Level students, suggesting an independent effect similar to 
two years of university. An interaction effect shows the advantage of having taken Inquiry 
1SS3 is more significant among Lower-Level students. This relates to skill development of 
the comparison group in this area between Lower and Upper-Levels. The skill levels 
developed by the Inquiry students early are lasting and continue to develop between Lower 
and Upper Levels (Table 13). 
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Table 13. Holistic Assessment of Academic Research Skills 


Variable 

Source 

Estimated Marginal 
Means 

Non- Inquiry 

Inquiry 

Effect 

size 

ANOVA 

F p. 

Inquiry Exercises Overall 

Group 

55.5 

61.7 

0.489 

7.134 

0.008 


Lower 

50.8 

60.0 

0.739 

1.797 

0.181 


Upper 

60.3 

63.4 

0.241 

1.797 

0.181 

Inquiry Exercises Overall 

Level 

Lower 

Level 

55.4 

Upper 

Level 

61.8 

0.499 

7.846 

0.005 


Oral Presentation 

This exercise tested student's abilities at designing and conducting an effective oral 
presentation. Students were assigned to small work groups and given 20 minutes to 
prepare a five-minute oral presentation based on a list of key-words. Students then made 
oral presentations to the other members of their groups. Videotapes of 124 presentations 
were sent to volunteer Raters who were educators from universities across Canada. Each 
Rater 'blindly' scored approximately 60 presentations; three Raters independently rated 
each presentation. Inter-rater reliability is acceptable (Cronbach's Alpha, .823.). Raters 
provided a global percentage score for each presentation and scores on six components 
which were summed to create a single variable: Rating Summary. The results are 
presented in Table 14. 

Table 14. Oral presentations _ 

Variable Estimated Marginal MANOVA 

Means 



Source 

Non- 

Inauirv 

Inquiry 

Effect 

F 

p. 




size 



Global Score 

Group 

66.8 

68.7 

.331 

3.218 

.074 


Lower 

66.0 

69.3 

.558 

1.527 

.217 


Upper 

67.5 

68.1 

.103 

1.527 

.217 

Rating Summary of 6 Quality 
Questions 

Group 

60.3 

63.3 

.320 

3.002 

.084 


Lower 

58.1 

64.8 

.688 

3.996 

.046 


Upper 

62.5 

62.0 

-.049 

3.996 

.046 



Lower 

Level 

Upper 

Level 




Global Score 

Level 

67.6 

67.8 

-.030 

.029 

.865 

Rating Summary of 6 Quality 
Questions 

Level 

61.5 

62.2 

-.079 

.198 

.656 


By both global and summed measures of quality, Inquiry student presentations were scored 
meaningfully higher than those of the Comparison group. Interestingly, time at university 
has no effect in developing this skill area. We also looked individually at the component 
questions. 
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Table 15. Component questions of summed variable _ 

Variable Estimated Marginal MANOVA 

Means 



Source 

Non- 

Inquiry 

Inquiry 

Effect 

size 

F 

P- 

Effectiveness 

Group 

62.0 

66.0 

.391 

4.476 

.035 


Lower 

59.2 

67.6 

.736 

3.534 

.061 


Upper 

63.9 

64.4 

.044 

3.534 

.061 

Clarity 

Group 

66.0 

68.0 

.298 

2.600 

.108 


Lower 

63.6 

71.0 

.694 

4.647 

.032 


Upper 

67.7 

66.6 

-.100 

4.647 

.032 

Style 

Group 

58.0 

58.0 

.129 

.491 

.484 


Lower 

54.3 

60.3 

.499 

4.024 

.046 


Upper 

60.5 

57.6 

-.241 

4.024 

.046 

Organization 

Group 

64.0 

66.0 

.286 

2.403 

.122 


Lower 

61.2 

67.7 

.567 

2.337 

.127 


Upper 

65.7 

65.7 

.004 

2.337 

.127 

Creativity 

Group 

52.0 

54.0 

.229 

1.545 

.215 


Lower 

49.4 

55.3 

.459 

1.563 

.212 


Upper 

52.9 

52.9 

-.001 

1.563 

.212 

Enjoyability 

Group 

62.0 

66.0 

.283 

2.346 

.126 


Lower 

61.0 

66.7 

.512 

1.558 

.213 


Upper 

64.1 

64.7 

.052 

1.558 

.213 



Lower 

Upper 






Level 

Level 




Effectiveness 

Level 

63.4 

64.1 

-.061 

.117 

.732 

Clarity 


67.3 

67.2 

.009 

.003 

.959 

Style 


57.3 

59.1 

-.144 

.655 

.419 

Organisation 


64.5 

65.7 

-.105 

.349 

.555 

Creativity 


52.4 

52.9 

-.041 

.054 

.816 

Eniovabilitv 


63.9 

64.4 

-.048 

.074 

.786 


Inquiry presentations were rated meaningfully higher than Non-Inquiry ones in 
effectiveness, clarity, organisation and enjoyability, but not in style or creativity. No 
meaningful differences existed between Lower and Upper-Level students in any of the 
categories. Almost all of the differences between Inquiry and Non-Inquiry students occur 
only among Lower-Level students. Inquiry students are only better off for one or two years 
after taking the course. It appears that this skill set, clearly developed by taking Inquiry 
1SS3, is attenuated with time in an environment which does not develop or nurture these 
skills. 


Critical Reasoning and Team Performance Exercise 

This exercise, an adaptation of Nemiroff and Pasmore's classic simulation "Lost at Sea" 
(2003, 1975), measured students' abilities in critical reasoning (the process of evaluating 
and ranking information when students'synthesize new knowledge), participatory learning, 
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and self-evaluation. Students were placed in four to six member teams consisting of either 
Inquiry students or Non-Inquiry students. Each team member individually ranked the 
importance of fifteen survival items for an imagined scenario of being stranded on a raft 
following a boating accident. Students then worked as a team to reach a consensus ranking 
of the same fifteen items. Our adaptation involved a third individual ranking following the 
team ranking. Finally, students completed a questionnaire evaluating their own and their 
team's performance. 

We calculated the values of and differences between student pre-scores, team consensus 
score and student post-scores. Differences between pre-score and team consensus score 
represent a measure of group functioning; better functioning groups should make greater 
improvements than individuals or poorer functioning groups. We take the difference 
between the team consensus and individual post scores, to be a measure of satisfaction 
with the group consensus. Table 16 shows results only by study group as there were no 
differences between Lower and Upper -Level students, suggesting time spent at university 
has no effect in this area. There are, however, some independent effects related to study 
group membership. Inquiry students scored higher than Non-Inquiry students in this 
exercise in all three rankings related to critical reasoning. 


Table 16. Team Critical Reasoning Exercise _ 

Estimated Marginal ANOVA 

Means 


Scores 

Non- 

Inquiry 

Inquiry 

Effect 

size 

F 

P- 

Individual Pre-score 

42.8 

46.6 

.283 

2.391 

.125 

Team Consensus Score 

51.7 

54.4 

.239 

1.488 

.225 

Individual Post-score 

49.5 

54.2 

.389 

3.950 

.049 

Differences 

Team score - Pre-score 

9.3 

8.1 

-.084 

0.183 

0.669 

Pre-score - Post-score 

7.1 

7.9 

.055 

0.078 

0.781 

Post-score - Team score 

-2.2 

-0.2 

-.350 

3.193 

0.077 

Absolute Difference of 
Post-score - Team score 

4.4 

4.0 

.085 

0.187 

0.666 


It appears that the consensus building exercise is generally effective in moving toward the 
correct answer set as scores on average were better for teams than for individuals. However 
both groups improved about the same amount between the individual and team scores, and 
thus there is no evidence for a group difference in team and or participatory learning skills. 
Likewise, we did not find meaningful differences between Inquiry and Non-Inquiry students 
in our absolute measure of change from the team ranking to the post ranking exercise. 
However, Non-Inquiry students scored slightly worse by departing from their consensus 
scores, Inquiry students did not. This suggests that while Inquiry students are no more 
confident in their team's consensus work, they benefited as individuals more from being in 
the group, an advantage also reflected in the increased advantage for Inquiry students in 
post-scores. Students' self-evaluation of individual and team performance showed little 
differences by study group or level. 
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Discussion 

We are mindful of the limits of quasi-experimental research especially regarding the issue of 
student self-selection into the Inquiry 1SS3 course. We also recognize that as faculty 
investigators we have a vested interest in the success of our intervention. In our analysis 
we have identified a number of potential limitations; we ask readers to share our caution in 
considering the study's results. We wish to make four observations from this research: it is 
feasible to see and measure the effects of relatively small educational interventions on 
students' abilities even after several years; university curriculum may not develop the skills 
many faculty members and students assume it is developing; a single, one-semester course 
can have relatively far reaching effects in academic skill development; and these skills can 
be lasting. 

Measuring Changes in Student's Abilities 

This study demonstrates that it is possible to detect changes in students' academic skills 
and abilities with a quasi-experimental methodology and a relatively small sample, using 
instruments designed to test actual differences in skill performance as well students' 
appraisals and self-reports. The instruments detected not only small and meaningful 
differences between Lower-Level and Upper-Level students, representing about two years 
of time spent at university, but also differences related to the taking of a one three unit 
first-year course. Some study participants took 1SS3 five years prior to our tests and 
Inquiry was only one of forty total university courses. The methodology detected 
differences between Inquiry and Non-Inquiry students in self-reported attitudes, 
experiences and activities, and in skill-related performance. 

Two methodological points are worth making. First, by and large, directly tested skills 
showed more meaningful differences than could be seen in the self-reported questionnaire 
data. This may suggest that evaluations of educational interventions that rely on 
questionnaire data alone may not be as sensitive as methodological designs that directly 
measure student learning. Second, holistic ratings can be different than the sum of the 
parts. For example, despite quite mixed results on some components of academic research 
skills, when the Raters considered the tasks holistically, Inquiry students appeared to have 
clearly performed better than the comparisons. 

Our sample included students who self-selected to either take or not take Inquiry 1SS3. 

This raised the possibility that differential effects in ability are artifacts of pre-existing 
differences between students. However, the method allowed us to draw conclusions about 
this likelihood based on analysis of the overall pattern of correspondence between Inquiry 
seminar foci and measured skill levels related to group membership on the one hand and 
to time spent in university on the other. For example, the fact that Inquiry students 
perceive themselves to have developed in two areas Understanding Other Cultures and 
Social Awareness, areas in which time spent at university has no independent effect and a 
slight independent effect respectively, links this enhanced perception of skill development 
to Inquiry 1SS3, which took these areas as thematic content. A second example is 
correspondence between inquiry group performance and Inquiry seminar foci in the skills 
of finding research materials. In sum, this research supports the feasibility of conducting 
effective evaluative research documenting the long-term effects of a wide variety of 
educational interventions. 

University Curriculum May Not Address the Skills It Is Assumed to Develop 

Although an unintended consequence of the research design, and a result of our controlling 
for differences between students with more or less exposure to university, we discovered a 
pattern of differences and similarities in skill levels between Lower-Level and Upper-Level 
students. With notable exceptions, Upper-Level students tend to have meaningfully higher 
skills than Lower-Level students. Taking university courses positively effects student 
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perceptions of their skill development and actual measured performance in skill areas 
including: academic research skills as a whole and specific skills including question 
development, accessing relevant information, summarizing a research paper and self- 
assessing performance. These improvements reinforce the "standard" wisdom that taking 
university courses increases academic skills and abilities in at least some areas. On the 
other hand, attending university seems to have little to no effect in improving student's 
abilities in oral presentations, critical reasoning, participatory learning, or in critical thinking 
and research planning. Most instructors would expect to see general progress in student 
skill levels, so the areas in which no effect was found are disturbing. We cannot easily 
generalize beyond our particular university context; but it is notable that the skills which 
showed little or no development tend to be those which are assumed to be naturally 
developed in most university programs - those related to critical thinking, oral 
communication, and collaborative or independent learning. This study suggests that these 
assumptions cannot be taken for granted. 

A Single Semester Inquiry Course Can Have Meaningful Effects on Skill 
Development 

This study demonstrates that the positive effects associated with taking Inquiry 1SS3 are 
significant. As the effects of the course are less obvious with the passage of time, the short 
term effects associated with the seminar can be most clearly seen by considering just Lower- 
Level students. Amongst this moiety, Inquiry students - those who on average had taken the 
course within two years of testing - there are widespread and meaningful advantages over 
the Lower-Level comparison Non-Inquiry students in most of the skill areas 
we directly tested. 

Lower Level Inquiry students perceived themselves to be more self-directed and less guided 
by others in their learning, and had a higher appraisal of their skill development than the 
Comparison group. Lower Level Inquiry students scored higher than the Comparison group 
in the academic research skills exercise rated as a whole and in many of the component 
skills including accessing relevant information, summarizing a research paper, planning 
research and self-assessing performance. Lower Level Inquiry students also performed 
better in oral presentations and in the skills related to the critical reasoning exercise. There 
were several areas in which Lower Level Inquiry students were no different than the 
Comparison group including question development, critical thinking, and teamwork. 

However, it should be emphasized that among Lower-Level students, Non-Inquiry students 
were never at an advantage over Inquiry students, with the single exception that they 
offered more and better justifications for their research questions. 
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Figure 3: Typical Pattern, exemplified by Holistic Rating of Academic Research Skills 



Level 


Group 

-Non-Inquiry 

- Inquiry 


Skills, Once Developed, Are Typically Lasting 

The situation becomes more complex when Upper Level students are added back into the 
mix and we look at effects related to a course taken between three and five years 
previously. Considering for the moment only Inquiry student scores, and comparing Lower 
and Upper levels, there are examples of both continued skill development and, in a few 
cases, skill loss. However, in sum the evidence indicates that the skills gained by students 
who took Inquiry 1SS3 are at least maintained between Lower and Upper Levels. 

Though the Inquiry student skills are lasting, the advantage they have over the Comparison 
group typically becomes less pronounced or disappears over time. This is not true in all 
cases but the pattern is common enough that we regard it as typical (see Figure 3). Inquiry 
students who have more recently taken the seminar have a clear advantage over the 
comparisons but with the passage of time do not progress as quickly as the comparisons. 

We are unable to fully explain this pattern. One possibility is that there are skills ceilings 
which Inquiry students simply reach earlier through seminar participation. The ceilings could 
relate to either student capacity or to 'performance expectation' determined by the 
demands of the university environment such that once a student has sufficiently mastered 
a skill set to meet the expectation of the environmental demands, there is little incentive 
for further progress. The fact that skills do not typically develop further nor dissipate among 
Inquiry students between lower and upper levels may suggest that the university 
environment is supportive enough to maintain the skills developed in Inquiry 1SS3 but not 
challenging enough to stimulate their further development. The relationship of skill ceilings 
to the supportiveness or challenge of the learning environment remains an important 
question for further inquiry. 


Conclusions 

The logic model (Astin, 1991, 1993) underlying this research assumes that a learning 
environment is a complex system in which many input and environment variables can 
potentially affect the (output) development of students' skills . While it is impossible to 
control for all confounding effects (for example, the differences between instructors' style 
of teaching) or to know the way those effects interact, it is possible to make reasonable 
connections between the inputs, environment and outputs based on evidence provided by 
this study. 
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Important questions about the potency of an Inquiry seminar are addressed by this 
research. The study supports the idea that it is possible to meaningfully develop academic 
and intellectual skills with a single first-year inquiry seminar course. The findings suggest 
that a little intervention (a three unit course) can have relatively large effects. For example, 
in many cases where 1SS3 is associated with higher performance, the magnitude and 
significance of the difference is similar to the difference between Lower and Upper-Level 
students, roughly a two year advantage. 

The results outlined here have significant implications for the way universities organize and 
develop their curriculum, especially for first-year students. These findings suggest that 
offering a first-year inquiry seminar can be a useful activity for students and the university. 
At the same time we need to know more about the relative importance of courses that focus 
on the development of academic skills per se versus those that embed skill development in 
an engaging pedagogy such as inquiry-based learning. 

On the other hand, both direct efforts at teaching skills and the cumulative effects of 
attending university had little or no effect on performance measures of critical thinking, 
question development, and research planning. As van Gelder (2005) points out "almost 
everyone agrees that one of the main goals of education, at whatever level, is to help 
develop general thinking skills, particularly critical thinking skills." Our findings suggest 
that our university specifically, and other universities by extension, are failing at this task. 
This introduces a considerable challenge to universities. What steps must they take if they 
are in improving the abilities of students regarding critical thinking, question development 
and research planning? 

These are of course, complex abilities that, like a second language, require more than a 
single course or an overall environment supportive of these skill areas. This leads us to 
believe that more sustained attention needs to be paid to these skill sets throughout 
university curricula. If we expect students to be life-long learners, universities will need to 
commit the time and resources to supporting this goal. We hypothesize that extension of 
inquiry style courses oriented to supporting and challenging the further development of 
academic and intellectual skills throughout the curricula would ensure universities do in 
fact produce life-long learners and inquirers. 

It is also worthy of note that our findings indicate that measuring skills directly is more 
sensitive than collecting data from self reports, suggesting that students may not be fully 
aware of the skills they have developed. The inquiry seminar described in this paper both 
raised skill levels and awareness of skill levels. The literature suggests that raising 
awareness of skills is important in facilitating their adaptation to different learning situations 
(Shepherd, 2000) which we believe is an important aspect of lifelong learning. It may be 
that one of the important things university can do is make students self-aware of the skills 
they possess, both the ones they come with and those they develop while at university. 
These changes would provide students with a heightened awareness of the value of a 
university education and motivation to continue the process of life-long learning. 
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